The deterministic annealing approach for discriminative continuous HMM design

We propose a deterministic annealing (DA) algorithm to design classifiers based on continuous observation hidden Markov models. The algorithm belongs to the class of minimum classification error (MCE) techniques that are known to outperform maximum likelihood (ML) design. Most MCE methods smooth the piecewise constant classification error cost to facilitate the use of local descent optimization methods, but are susceptible to the numerous shallow local minimum traps that riddle the cost surface. The DA approach employs randomization of the classification rule followed by minimization of the corresponding expected misclassification rate, while controlling the level of randomness via a constraint on the Shannon entropy. The effective cost function is smooth and converges to the MCE cost at the limit of zero entropy. The proposed algorithm significantly outperforms both standard ML and standard MCE design methods on the E-set database.